Augmented reality event switching

ABSTRACT

A processing system including at least one processor may identify a user viewpoint of a user at a first venue, match a viewpoint at a second venue to the user viewpoint of the user at the first venue, detect a trigger condition to provide visual content of the second venue to the user at the first venue, obtain the visual content of the second venue, wherein the visual content of the second venue is obtained from the viewpoint at the second venue, and provide the visual content of the second venue to an augmented reality device of the user at the first venue, where the augmented reality device presents the visual content of the second venue as a visual overlay within a field of view of the user.

This application is a continuation of U.S. patent application Ser. No.17/461,240, filed on Aug. 30, 2021, which is herein incorporated byreference in its entirety.

The present disclosure relates generally to augmented reality devicesand systems, and more particularly to methods, computer-readable media,and apparatuses for obtaining and providing visual content of a secondvenue to an augmented reality device of a user at a first venue whereinthe visual content of the second venue is obtained from a viewpoint atthe second venue that is matched to a user viewpoint of the user at thefirst venue, and to methods, computer-readable media, and apparatusesfor presenting visual content of a second venue as a visual overlaywithin a field of view of a user wherein the visual content is from aviewpoint at the second venue that is matched to a user viewpoint at afirst venue.

BACKGROUND

Augmented reality (AR) and/or mixed reality (MR) applications and videochat usage are increasing. In one example, an AR endpoint device maycomprise smart glasses with AR enhancement capabilities. For example,the glasses may have a screen and a reflector to project outlining,highlighting, or other visual markers to the eye(s) of a user to beperceived in conjunction with the surroundings. The glasses may alsocomprise an outward facing camera to capture video of the physicalenvironment from a field of view in a direction that the user islooking, which may be used in connection with detecting various objectsor other items that may be of interest in the physical environment,determining when and where to place AR content within the field of view,and so on.

SUMMARY

In one example, the present disclosure describes a method,computer-readable medium, and apparatus for obtaining and providingvisual content of a second venue to an augmented reality device of auser at a first venue wherein the visual content of the second venue isobtained from a viewpoint at the second venue that is matched to a userviewpoint of the user at the first venue. For instance, in one example,a processing system including at least one processor may identify a userviewpoint of a user at a first venue, match a viewpoint at a secondvenue to the user viewpoint of the user at the first venue, detect atrigger condition to provide visual content of the second venue to theuser at the first venue, obtain the visual content of the second venue,wherein the visual content of the second venue is obtained from theviewpoint at the second venue, and provide the visual content of thesecond venue to an augmented reality device of the user at the firstvenue, where the augmented reality device presents the visual content ofthe second venue as a visual overlay within a field of view of the user.

In another example, the present disclosure describes an apparatus,computer-readable medium, and method for presenting visual content of asecond venue as a visual overlay within a field of view of a userwherein the visual content is from a viewpoint at the second venue thatis matched to a user viewpoint at a first venue. For instance, anapparatus comprising a processing system including at least oneprocessor (e.g., an augmented reality device of a user) may detect atrigger condition to provide a visual content of a second venue to theuser at a first venue, identify a user viewpoint of the user at thefirst venue, obtain the visual content of the second venue, where thevisual content of the second venue is obtained from a viewpoint at thesecond venue that is matched to the user viewpoint of the user at thefirst venue, and present the visual content of the second venue as avisual overlay within a field of view of the user.

BRIEF DESCRIPTION OF THE DRAWINGS

The teaching of the present disclosure can be readily understood byconsidering the following detailed description in conjunction with theaccompanying drawings, in which:

FIG. 1 illustrates an example system related to the present disclosure;

FIG. 2 illustrates an example of detecting visual markers at a venue, inaccordance with the present disclosure;

FIG. 3 illustrates an example view (which may be a non-augmented realityview) of a venue during an event, in accordance with the presentdisclosure;

FIG. 4 illustrates an example view representing the visual experience ofa user at a venue who is physically present during a live in-personevent, but who is currently experiencing AR content from another venuevia an AR device of the user, in accordance with the present disclosure;

FIG. 5 illustrates an alternate example view, which may be similar tothe view of FIG. 4 , but in which a virtual scoreboard is presented inaddition to augmented reality content in an overlay;

FIG. 6 illustrates a flowchart of an example method for obtaining andproviding visual content of a second venue to an augmented realitydevice of a user at a first venue wherein the visual content of thesecond venue is obtained from a viewpoint at the second venue that ismatched to a user viewpoint of the user at the first venue;

FIG. 7 illustrates a flowchart of an example method for presentingvisual content of a second venue as a visual overlay within a field ofview of a user wherein the visual content is from a viewpoint at thesecond venue that is matched to a user viewpoint at a first venue; and

FIG. 8 illustrates an example high-level block diagram of a computingdevice specifically programmed to perform the steps, functions, blocks,and/or operations described herein.

To facilitate understanding, identical reference numerals have beenused, where possible, to designate identical elements that are common tothe figures.

DETAILED DESCRIPTION

Examples of the present disclosure describe methods, computer-readablemedia, and apparatuses for obtaining and providing visual content of asecond venue to an augmented reality device of a user at a first venuewherein the visual content of the second venue is obtained from aviewpoint at the second venue that is matched to a user viewpoint of theuser at the first venue and methods, computer-readable media, andapparatuses for presenting visual content of a second venue as a visualoverlay within a field of view of a user wherein the visual content isfrom a viewpoint at the second venue that is matched to a user viewpointat a first venue. In particular, examples of the present disclosureenable a user to have an in-person experience and a simultaneous, ortime-shifted, simulated experience by presenting content from a secondvenue as augmented reality (AR) content while the user is at a firstvenue that creates the appearance of viewing the event at the secondvenue as though it were occurring at the first venue. In one example,the present disclosure utilizes the user's orientation and environmentin the first venue to simulate the second event as though it wereoccurring at the first venue.

In one example, a user may be physically present at Venue 1 observingEvent 1. While present there, a simulation of Event 2 in Venue 2, whichis either occurring at the same time, or a previous time, may bepresented to the user. In other words, the first and second events maybe occurring in real-time, e.g., at the same time, or the AR content ofthe second event may be time-shifted. In an illustrative example, a usermay be a spectator physically present at a football game, baseball game,or other sporting events. However, it should be noted that in oneexample, the user may be physically present at Venue 1 at which time noparticular event is taking place within or on a performance area ofVenue 1 (e.g., on a field of play), while experiencing AR content fromVenue 2. Other examples may include other types of events at other typesof venues, such as concert halls and theaters.

To illustrate, the user may be physically present at a first venue inLos Angeles watching a football game. At certain points during the gameat the Los Angeles venue, the user may view, using an augmented reality(AR) device, content related to an event at an Atlanta venue (e.g.,another football game). The augmented reality content may be positionedso that it simulates a seamless experience for the user. That is, theuser's view of the event at the second venue is as though the user isviewing it from the same perspective and field of view that is availablefrom the user's position at Venue 1.

In one example, the user's AR device may also be equipped with a videocamera. In one example, the camera may be used to detect expecteddemarcation landmarks in the user's field of view, such as fieldmarkings, numbers, or lines, goal posts, fences, or the like. Thesemarkers, or landmarks may comprise a “map” of the user's view, and maybe used to represent the user's perspective, or viewpoint, in viewingthe event at Venue 1. Alternatively, or in addition, a viewingperspective for each seat or seating section may be known in advance, inwhich case, the user's view/viewpoint may be known as long as the user'sseating location or other position/location information is accessible.This map may be sent by the AR device to an AR content server, or in theevent that the AR device does not have an independent networkconnection, by the AR device via a network connected endpoint device ofthe user having a local connection to the AR device. The AR contentserver may associate a data representation of this map with the user ina subscriber database.

At the same time, or at a previous time, an event, such as anotherfootball game, may occur at Venue 2. Venue 2 may be equipped with aplurality of video cameras and microphones, positioned throughout thevenue. Each camera may be positioned so as to avoid obstructions of afield of view, while also capturing a spectator's field of view of theevent and audio from the same location. In one example, each cameraposition at Venue 2 may be a known location/known viewpoint, e.g., inrelation to one or more markers of an area of performance, such as adistance and bearing to one or more markers of a center of a field, orthe like. Alternatively, or in addition, each camera may capture its ownvenue view map, or may provide images or video to the AR content server,which may determine the markers/landmarks within the field of view,e.g., in the same manner as described above. In one example, a datarepresentation of the venue view map for each camera position at Venue 2is stored in a venue mapping database. The data representation of thecamera venue view map uses the same format as the user's venue view mapat Venue 1.

At a point in time, the user's view at Venue 1 may be augmented so as topresent AR content of an event at Venue 2 as though the event iscurrently occurring at Venue 1. In one example, the user may bepresented with options of multiple alternative venues offering ARcontent, for instance, via an application (app) running on the user's ARdevice that is in communication with the AR content server (or via theuser's endpoint device in communication with the AR device and the ARcontent server). The app may be responsive to the user's request toswitch from one event to another. This may be accomplished via a tactileinput or spoken input/voice command for example.

Upon receiving a request to switch to event 2 (e.g., to display ARcontent from Venue 2 and/or event 2), the AR content server may retrievethe view map for the user from the subscriber database and compare itwith available seating view maps from the venue mapping database for therequested alternate venue (e.g., Venue 2). In this way, the user'scurrent seating position at Venue 1 is compared to available cameraviews at Venue 2 and the closest match camera view to the user'sposition in Venue 1 is identified. When the closest match camera inVenue 2 is identified, AR content associated with that camera may beretrieved from the AR content database. The AR content may comprisevideo from the camera, audio from an accompanying microphone, or visualcontent extracted from the video (e.g., using video processing andediting to identify players, performers, and/or objects relevant to anevent (e.g., goals, sticks, balls, etc.), distinguishing frombackground, and extracting these portions of the visual imagery), etc.

The AR content from the identified best match camera of Venue 2 may besent by the AR content server to the user AR device. In one example,based on the user's view in Venue 1, portions of the view may beidentified for use in AR display/projection. For instance, the area of afield, an area of a scoreboard, and so forth may be available. Using theavailable portions of the view, in one example, the portion of the ARcontent from Venue 2 that corresponds to the available portion(s) fromthe user's view at Venue 1 may be presented to the user via the user'sAR device. In one example, the AR content may be presented as a dominantoverlay such that the user can mostly pay attention to AR content fromVenue 2, but also such that that real-world imagery of Venue 1 is notcompletely obstructed. For instance, the AR content may appear astransparent (but dominant) imagery via angled projection on a glass orsimilar screen within the user's field of view.

In one example, available portions of the user's view identified for usein AR display/projection may also be included in the view map, and theAR server may further select AR content matching the available area(s).Alternatively, or in addition, AR content comprising a field view may bepresented as AR content, while a virtual scoreboard (or otherinformational blocks, for events of various types) may be generatedusing available informational data for the event. In this manner, theuser may see simultaneous elements within the user's field of view—areal-life view, a segment of AR content to present the field view fromVenue 2, and a created AR image using the additional informational data.

In one example, when the user invokes a venue switch, the user mayspecify the duration of the switch in advance. For example, the switchmay be defined for a duration that only lasts as long as a timeout(e.g., a team calling a timeout during the game, or a commercial breakevent) during an event at Venue 1 where the user is physically present.In one example, the end of the timeout may be determined by the ARcontent server monitoring a real-time informational data source for thegame. Alternatively, the user may speak a command such as: “Show me theother game until something exciting happens here.” The AR content servermay monitor a number of factors to determine a level of excitement,including ambient noise level of an audio stream from the venue, may usean analysis of commentary from one or more broadcast announcers, ananalysis of social media activity levels for postings related to thegame, a visual analysis (e.g., using video signature(s) of “start ofplay,” or the like), and so on.

Similarly, in one example, the AR content server may monitor the statusof the game at Venue 1 to detect opportunities to offer a venue viewswitch. For example, if a timeout, halftime, or other stops in action isdetermined by monitoring a real-time feed of game information, or viaaudio or video analysis, the AR content server may present an offer tothe user to switch to another venue (or to automatically switch toanother venue, e.g., Venue 2, if already selected by the user and/orpreviously experienced during a current session). In one example, the ARcontent server may present choices of other venue views that areavailable, and may optionally calculate an “excitement level”/score foreach alternate venue and prioritize offers for the most exciting eventsfirst. The excitement level/score may be calculated by the AR contentserver using ambient noise levels, social media activity, timeremaining, score, probability/odds of either team winning, importance ofother games impacting the standing of a current team playing in Venue 1(e.g., outcomes of other games impacting playoff chances of teamscurrently playing in Venue 1), and/or other factors.

When a venue view shift occurs, the AR content server may also, inconjunction with the presentation of visual AR content for the alternatevenue (e.g., Venue 2), provide audio associated with the matchinglocation of the alternate venue. For instance, the user may also haveearphones to present the audio as it is streamed as a real-time feed. Assuch, the user may hear the ambiance as if the user is at thecorresponding location/position in Venue 2. In one example, the user'sapp may include a social experience function. For instance, User 1 mayuse the app to have a voice conversation or send voice or text messagesto a friend, User 2. User 1 may, for instance, invite User 2 to meet atan alternate virtual venue. For example, both User 1 and User 2 may thenswitch their venue views so that they both experience the switched viewexperience using their own seating perspectives while continuing a voiceconversation. Thus, although their AR views may be from differentseating perspectives, they may converse as though they were sittingtogether.

In another example, a user may be practicing at a first venue and mayexperience an AR simulation of another venue. For instance, a baseballplayer may be taking batting practice using an AR headset to projectimagery from another stadium. In one example, the imagery from the otherstadium may also include a live or recorded projection of player in thefield who may chase down fly balls and grounders. This may betterprepare the batter for playing in the other stadium. Similarly, inanother example, a soccer player may be practicing penalty kicks at afirst venue and may use an AR headset to simulate the goal, stadium,crowd, etc. from another stadium where the player has an upcoming match.These and other aspects of the present disclosure are discussed ingreater detail below in connection with the examples of FIGS. 1-8 .

To further aid in understanding the present disclosure, FIG. 1illustrates an example system 100 in which examples of the presentdisclosure may operate. The system 100 may include any one or more typesof communication networks, such as a traditional circuit switchednetwork (e.g., a public switched telephone network (PSTN)) or a packetnetwork such as an Internet Protocol (IP) network (e.g., an IPMultimedia Subsystem (IMS) network), an asynchronous transfer mode (ATM)network, a wireless network, a cellular network (e.g., in accordancewith 3G, 4G/long term evolution (LTE), 5G, etc.), and the like relatedto the current disclosure. It should be noted that an IP network isbroadly defined as a network that uses Internet Protocol to exchangedata packets. Additional example IP networks include Voice over IP(VoIP) networks, Service over IP (SoIP) networks, and the like.

In one example, the system 100 may comprise a network 102, e.g., atelecommunication service provider network, a core network, anenterprise network comprising infrastructure for computing andcommunications services of a business, an educational institution, agovernmental service, or other enterprises. The network 102 may be incommunication with one or more access networks 120 and 122, and theInternet (not shown). In one example, network 102 may combine corenetwork components of a cellular network with components of a tripleplay service network; where triple-play services include telephoneservices, Internet services and television services to subscribers. Forexample, network 102 may functionally comprise a fixed mobileconvergence (FMC) network, e.g., an IP Multimedia Subsystem (IMS)network. In addition, network 102 may functionally comprise a telephonynetwork, e.g., an Internet Protocol/Multi-Protocol Label Switching(IP/MPLS) backbone network utilizing Session Initiation Protocol (SIP)for circuit-switched and Voice over Internet Protocol (VoIP) telephonyservices. Network 102 may further comprise a broadcast televisionnetwork, e.g., a traditional cable provider network or an InternetProtocol Television (IPTV) network, as well as an Internet ServiceProvider (ISP) network. In one example, network 102 may include aplurality of television (TV) servers (e.g., a broadcast server, a cablehead-end), a plurality of content servers, an advertising server (AS),an interactive TV/video on demand (VoD) server, and so forth.

In accordance with the present disclosure, application server (AS) 104may comprise a computing system or server, such as computing system 800depicted in FIG. 8 , and may be configured to provide one or moreoperations or functions for obtaining and providing visual content of asecond venue to an augmented reality device of a user at a first venuewherein the visual content of the second venue is obtained from aviewpoint at the second venue that is matched to a user viewpoint of theuser at the first venue, such as illustrated and described in connectionwith the example method 600 of FIG. 6 . It should be noted that as usedherein, the terms “configure,” and “reconfigure” may refer toprogramming or loading a processing system withcomputer-readable/computer-executable instructions, code, and/orprograms, e.g., in a distributed or non-distributed memory, which whenexecuted by a processor, or processors, of the processing system withina same device or within distributed devices, may cause the processingsystem to perform various functions. Such terms may also encompassproviding variables, data values, tables, objects, or other datastructures or the like which may cause a processing system executingcomputer-readable instructions, code, and/or programs to functiondifferently depending upon the values of the variables or other datastructures that are provided. As referred to herein a “processingsystem” may comprise a computing device including one or moreprocessors, or cores (e.g., as illustrated in FIG. 8 and discussedbelow) or multiple computing devices collectively configured to performvarious steps, functions, and/or operations in accordance with thepresent disclosure.

Thus, although only a single application server (AS) 104 is illustrated,it should be noted that any number of servers may be deployed, and whichmay operate in a distributed and/or coordinated manner as a processingsystem to perform operations for detecting and modifying actions ofvisual representations of users in visual content, in accordance withthe present disclosure. In one example, AS 104 may comprise an ARcontent server as described herein. In one example, AS 104 may comprisea physical storage device (e.g., a database server), to store varioustypes of information in support of systems for obtaining and providingvisual content of a second venue to an augmented reality device of auser at a first venue wherein the visual content of the second venue isobtained from a viewpoint at the second venue that is matched to a userviewpoint of the user at the first venue, in accordance with the presentdisclosure. For example, AS 104 may store object detection and/orrecognition models, user data (including user device data), and so forththat may be processed by AS 104 in connection with selecting andproviding visual content, e.g., AR content from remote venues to a usera particular physical venue, or that may be provided to devicesobtaining such AR content via AS 104. For ease of illustration, variousadditional elements of network 102 are omitted from FIG. 1 .

In one example, the access network(s) 120 may be in communication with adevice 131. Similarly, access network(s) 122 may be in communicationwith one or more devices, e.g., device 141. Access networks 120 and 122may transmit and receive communications between devices 131 and 141,between devices 131 and 141, and application server (AS) 104, othercomponents of network 102, devices reachable via the Internet ingeneral, and so forth. In one example, each of devices 131 and 141 maycomprise any single device or combination of devices that may comprise auser endpoint device. For example, the devices 131 and 141 may eachcomprise a mobile device, a cellular smart phone, a wearable computingdevice (e.g., smart glasses) a laptop, a tablet computer, a desktopcomputer, an application server, a bank or cluster of such devices, andthe like. In one example, devices 131 and 141 may each compriseprograms, logic or instructions for presenting visual content of asecond venue as a visual overlay within a field of view of a userwherein the visual content is from a viewpoint at the second venue thatis matched to a user viewpoint at a first venue. For example, devices131 and 141 may each comprise a computing system or device, such ascomputing system 800 depicted in FIG. 8 , and may be configured toprovide one or more operations or functions for presenting visualcontent of a second venue as a visual overlay within a field of view ofa user wherein the visual content is from a viewpoint at the secondvenue that is matched to a user viewpoint at a first venue, such asillustrated and described in connection with the example method 700 ofFIG. 7 .

In one example, the access networks 120 and 122 may comprise DigitalSubscriber Line (DSL) networks, public switched telephone network (PSTN)access networks, broadband cable access networks, Local Area Networks(LANs), wireless access networks (e.g., an IEEE 802.11/Wi-Fi network andthe like), cellular access networks, 3rd party networks, and the like.For example, the operator of network 102 may provide a cable televisionservice, an IPTV service, or any other types of telecommunicationservice to subscribers via access networks 120 and 122. In one example,the access networks 120 and 122 may comprise different types of accessnetworks, may comprise the same type of access network, or some accessnetworks may be the same type of access network and others may bedifferent types of access networks. In one example, the network 102 maybe operated by a telecommunication network service provider. The network102 and the access networks 120 and 122 may be operated by differentservice providers, the same service provider or a combination thereof,or may be operated by entities having core businesses that are notrelated to telecommunications services, e.g., corporate, governmental oreducational institution LANs, and the like. For instance, in oneexample, one of the access network(s) 120 may be operated by or onbehalf of a first venue (e.g., associated with first physicalenvironment 130). Similarly, in one example, one of the accessnetwork(s) 122 may be operated by or on behalf of a second venue (e.g.,associated with second physical environment 140).

In one example, the device 131 is associated with a first user (user 1)191 at a first physical environment 130. As illustrated in FIG. 1 , thedevice 131 may comprise a wearable computing device (e.g., smart glassesor smart goggles) and may provide a user interface 135 for user 191. Forinstance, device 131 may comprise smart glasses with augmented reality(AR) enhancement capabilities. For example, endpoint device 131 may havea screen and a reflector to project outlining, highlighting, or othervisual markers to the eye(s) of user 191 to be perceived in conjunctionwith the surroundings. Device 131 may further comprise or may be coupledto earphones, a speaker, or the like, e.g., via wired or wireless link,which may be used to present AR content comprising audio data from othervenues, such as second physical environment 140. Similarly, device 131may further include a microphone for capturing audio of the firstphysical environment 130 from the location of user 191. In one example,device 131 may also comprise an outward facing camera to capture videoof the first physical environment 130 from a field of view shown on userinterface 135 in a direction that user 191 is looking. In the presentexample, device 131 may present an augmented reality (AR) overlay 137(e.g., within a user interface 135, which in the present example may becoextensive with the field of view of the user).

Also associated with physical environment 130 is a plurality of cameras132-134 which may capture videos of the first physical environment 130,e.g., including items, objects, or other aspects of the environment. Inone example, cameras 132-134 may provide video streams from variousviewpoints (e.g., locations) within the first physical environment 130to AS 104 (e.g., on an ongoing basis, or on demand). In one example,each of the cameras 132-134 may have an associated microphone (notshown) for capturing audio of the same location/viewpoint as theassociated camera. Thus, it should be noted that as described herein, ARcontent may further include audio of a same location/viewpoint ascorresponding video or other visual data derived therefrom that may beprovided to a user for presentation as AR content.

In one example, the device 131 may present visual content of one or moreother venues (e.g., other physical environments, such as second physicalenvironment 140) via user interface 135 (e.g., presented as overlay 137in FIG. 1 ). In one example, the physical environment 130 and userinterface 135 may comprise an augmented reality (AR) or a mixed reality(MR) environment, e.g., when the physical environment 130 remainsvisible to user 191 when using device 131, and the visual contentreceived from one or more other venues is presented spatially in anintelligent manner with respect to the physical environment 130. In oneexample, the components associated with user 191 and/or first physicalenvironment 130 that are used to establish and support a visualcommunication session may be referred to as a “communication system.”For instance, a communication system may comprise device 131, or device131 in conjunction with a smartphone or personal computer, a wirelessrouter, or the like supporting visual communication sessions of device131, device 131 in conjunction with separate headphones, earbuds, or thelike, and so on.

Similarly, device 141 may be associated with a second user (user 2) 192at a second physical environment 140. As illustrated in FIG. 1 , thedevice 141 may comprise a wearable computing device that is the same orsimilar to device 131. Device 141 may provide a user interface for user192 similar to that which is illustrated for user 191. However, forclarity of illustration, such user interface is omitted from FIG. 1 .Also associated with second physical environment 140 is a plurality ofcameras 142-144 which may capture videos of the second physicalenvironment 140, e.g., including items, objects, or other aspects of theenvironment. In one example, cameras 142-144 may provide video streamsfrom various viewpoints (e.g., locations) within the second physicalenvironment 140 to AS 104 (e.g., on an ongoing basis, or on demand).Cameras 142-144 may similarly have associated microphones for capturingaudio data of the second physical environment 140 from the locations ofthe respective cameras 142-144.

Devices 131 and 141 may measure, record, and/or transmit data related tomovement and position, such as locations, orientations, accelerations,and so forth. For instance, devices 131 and 141 may each include aGlobal Positioning System (GPS) unit, a gyroscope, a compass, one ormore accelerometers, and so forth. In one example, devices 131 and 141may include transceivers for wireless communications, e.g., forInstitute for Electrical and Electronics Engineers (IEEE) 802.11 basedcommunications (e.g., “Wi-Fi”), IEEE 802.15 based communications (e.g.,“Bluetooth”, “ZigBee”, etc.), cellular communication (e.g., 3G, 4G/LTE,5G, etc.), and so forth. Thus, for example, device 131 may providevarious measurements, requests/instructions, and so forth to AS 104(e.g., via access network(s) 120) and may similarly receive visualcontent of second physical environment 140 from AS 104 (and similarlyfor device 141).

In one example, AS 104 and device 131 may operate in a distributedand/or coordinated manner to perform various steps, functions, and/oroperations described herein. To illustrate, AS 104 may establish andmaintain a communication session with device 131 and may store andimplement one or more configuration settings specifying which visualcontent from which camera of second physical environment 140 to provideto device 131, any modifications of the visual content to be applied byAS 104, and so forth. The visual content may comprise video content,which may include visual imagery of a physical environment, and which insome cases may further include recorded audio of the physicalenvironment. In one example, the visual content may comprise a modifiedvideo content, or visual content extracted from a video obtained fromone of the cameras 142-144 of the second physical environment 140, suchas visual content comprising players and any equipment that areseparated from visual content comprising background or “unimportant”visual content (e.g., where “unimportant” visual content may beaccording to a defined list of visual content that may be excluded by AS104 and/or according to a defined list of “important” visual content,where any other content not on such a list may be segregated anddiscarded). Although the present examples are described primarily inconnection with user 191 and device 131, it should be understood thatthe same or similar operations or functions may similarly apply device141 and user 192. For instance, user 192 may obtain and experiencevisual content from first physical environment 130, e.g., from one ofthe cameras 132-134 having a location, or viewpoint within the firstphysical environment 130 that best matches a corresponding location, orviewpoint of user 192 within the second physical environment 140.

As used herein, the terms augmented reality (AR) environment may be usedherein to refer to the entire environment experienced by a user,including real-world images and sounds combined with generated imagesand sounds. The generated images and sounds added to the AR environmentmay be referred to as “virtual objects” and may be presented to usersvia devices and systems of the present disclosure. While the real worldmay include other machine generated images and sounds, e.g., animatedbillboards, music played over loudspeakers, and so forth, these imagesand sounds are considered part of the “real-world,” in addition tonatural sounds and sights such as other physically present humans andthe sound they make, the sound of wind through buildings, trees, etc.,the sight and movement of clouds, haze, precipitation, sunlight and itsreflections on surfaces, and so on.

As illustrated in FIG. 1 , user 191 may be physically present at thefirst physical environment 130 (e.g., a first venue) watching a firstevent (e.g., a football game). The field of view of the user 191 mayinclude other seats in a stadium, a field of play, and players on thefield engaging in the game. In this case, the players 199 may bephysically present on the field at the first physical environment 130and visible to user 191. At an illustrative time represented by theexample of FIG. 1 , user 191 may determine to “switch” to viewing asecond event, e.g., taking place at the second physical environment 140,as AR content obtained and presented by device 131. Alternatively, or inaddition, user 191 may provide device 131 and/or AS 104 with one or moretrigger conditions for switching to viewing a remote event (e.g., thesecond event). The trigger conditions may include stops in play,timeouts, intermissions, etc., which may be detected by audio or videocontent obtained by device 131 (e.g., via its own camera, microphone,and/or other sensors) and/or by AS 104 from any one or more of cameras132-134, any associated microphone(s), etc., from informational feed(s)associated with the first physical environment 130 and/or secondphysical environment 140, and so forth.

In any case, when it is determined to switch to viewing the second eventat second physical environment 140, AS 104 may obtain visual contentfrom one of cameras 142-144 having a location/viewpoint in the secondphysical environment 140 that is a best match to a location/viewpoint ofuser 191 (e.g., of device 131) within the first physical environment130. In one example, AS 104 may make the determination of which ofcameras 142-144 is a best match viewpoint prior to obtaining the visualcontent. Alternatively, AS 104 may make a prior determination of a bestmatch camera, and if the user 191 has not moved, or has not movedsubstantially from a prior location/viewpoint, then AS 104 may obtainthe visual content from the camera that has already been determined.

In one example, AS 104 and/or device 131 may use a venue map of viewinglocations, a GPS unit of device 131, an altimeter, etc., to determine aviewpoint/location of user 191 (and device 131). In addition, AS 104 mayalternatively or additionally use a venue map of camera (and microphone)positions within second physical environment 140 (and similarly forfirst physical environment 130) to determine the best match camera. Forinstance, a distance and bearing of the location of user 191 to aparticular locational marker of the first physical environment 130 maybe known or determined (e.g., a distance and bearing to a center of afield of play, a distance and bearing to a closest goal post, etc.), orsimilarly x,y vector component distances (or x,y,z where a height ofviewing and/or seating position is relevant). Similarly, a distance andbearing, or vector component distances of a camera with respect to asame marker at the second physical environment 140 may be known andrecorded in a venue map. Thus, the closest matching camera may be thatwhich is the closest (e.g., closest vector distance of the respectivecamera and user locations).

In this regard, it should be noted that in one example, the venue typesof the first physical environment 130 and second physical environment140 are known in advance and are the same (e.g., both comprise footballfields, which are generally standardized and include the same basicfeatures of 100 yards plus endzones, goalposts, typically markings forevery 10 yards and/or 5 yard marks, etc.). However, it should be notedthat there may be substantial variations beyond the relevant areas ofperformance. For instance, first physical environment 130 may comprise astadium of a professional football team having a large scoreboard,multiple levels/tiers of seating, etc., while the second physicalenvironment 140 may comprise a single level of bleachers and a field fora small college football team. Nevertheless, visual content relating tothe actual fields of play may be projected from one to the other as ARcontent for viewers such as user 191, etc. without regard to theperipheral settings. In one example (and as discussed in greater detailbelow), AS 104 and/or device 131 may alternatively or additionally usevisual analysis of video from cameras 142-144 and an outward facingcamera of device 131 to determine a best match camera at second physicalenvironment 140.

For illustrative purposes, it may be assumed that camera 143 isdetermined to have a best match viewpoint (e.g., a closest correspondinglocation/position). As noted above, the visual content may comprisevideo (e.g., a video feed) from camera 143. Alternatively, the visualcontent may comprise modified video, or visual content extracted fromsuch video. In the present example, the visual content may correspond tothe window 147. However, in one example, the window 147 may compriseless than all of a field of view, or “viewport” of camera 143. Forinstance, camera 143 may be capable of capturing all of window 145. Inone example, video comprising all of window 145 may be obtained by AS104 from camera 143, whereupon AS 104 may determine that window 147comprises the visual content that should be provided to device 131 andmay extract the visual content of window 147. In another example, camera143 may be adjusted such that the zoom is aligned to the window 147,e.g., to the field of play plus an additional region comprising adefined distance around the field, or the like. In the example of FIG. 1, the visual content from camera 143 may be provided by AS 104 to device131. Device 131 may then present the visual content as overlay 137. Inthis case, the visual content from camera 143 that is presented mayinclude the players 198 made visible within the portion of the field ofview of user interface 135 comprising overlay 137.

In one example, the visual content from camera 143 presented via device131 (e.g., the “AR content”) may include all of the video obtained fromwindow 147. In this case, imagery of the entire field at the secondphysical environment 140, the players on the field, the goalposts, etc.may all be part of the visual content that is provided to and presentedby device 131 within overlay 137. In another example, the imagery ofplayers 198 may be segregated from the imagery of the field and otherbackground within video from camera 143, such that only the playerimages are part of the visual content presented via overlay 137. In anycase, the AR content presented in overlay 137 may be presented as a“dominant” overlay, e.g., with a transparency that is selected with theintention that within the region of the field of view of the userinterface 135 comprising the overlay 137, user 191 will primarilyperceive the AR content, while the real-world aspects of the firstphysical environment 130 may remain slightly visible in the coincidentportions of overlay 137. However, in one example, the transparency maybe adjustable by user 191 via command or input to device 131, may beadjustable by device 131 in response to certain conditions, such aschanging ambient light levels, sunlight, glare, etc. at first physicalenvironment 130, any emergencies or other defined occurrences, such asdetection of user 191 attempting to walk or otherwise move around,another person present within one meter in front of device 131, anotherperson talking to user 191 from a position in front of device 131, orthe like, which may be detected by device 131 (such as using machinelearning (ML)-based detection models applied to video from an outwardfacing camera of device 131, audio from a microphone, position andorientation data from a gyroscope, compass, GPS unit, etc.), and soforth.

In one example, AS 104 may store detection models that may be applied byAS 104 or deployed to user devices (such as device 131), in order todetect items of interest in video from cameras 132-134, cameras 142-144,outward facing cameras of devices 131 and 141, etc. For instance, inaccordance with the present disclosure, the detection models may bespecifically designed for detecting fields, field lines, goals, fences,or other expected portions of playing areas, for detecting players,balls, bats, sticks, pucks, or other equipment, scoreboards, viewingboxes, etc. (and similarly for stages, instruments, performers, podiums,set pieces, and so forth for other types of events). The MLMs, orsignatures, may be specific to particular types of visual/image and/orspatial sensor data, or may take multiple types of sensor data asinputs. For instance, with respect to images or video, the input sensordata may include low-level invariant image data, such as colors (e.g.,RGB (red-green-blue) or CYM (cyan-yellow-magenta) raw data (luminancevalues) from a CCD/photo-sensor array), shapes, color moments, colorhistograms, edge distribution histograms, etc. Visual features may alsorelate to movement in a video and may include changes within images andbetween images in a sequence (e.g., video frames or a sequence of stillimage shots), such as color histogram differences or a change in colordistribution, edge change ratios, standard deviation of pixelintensities, contrast, average brightness, and the like. For instance,these features could be used to help quantify and distinguish plasticseats from a concrete floor, metal railings, etc. In one example, thedetection models may be to detect particular items, objects, or otherphysical aspects of an environment (e.g., field lines definingboundaries of an area of play, or within the area of play).

In one example, MLMs, or signatures, may take multiple types of sensordata as inputs. For instance, MLMs or signatures may also be providedfor detecting particular items based upon LiDAR input data, infraredcamera input data, and so on. In accordance with the present disclosure,a detection model may comprise a machine learning model (MLM) that istrained based upon the plurality of features available to the system(e.g., a “feature space”). For instance, one or more positive examplesfor a feature may be applied to a machine learning algorithm (MLA) togenerate the signature (e.g., a MLM). In one example, the MLM maycomprise the average features representing the positive examples for anitem in a feature space. Alternatively, or in addition, one or morenegative examples may also be applied to the MLA to train the MLM. Themachine learning algorithm or the machine learning model trained via theMLA may comprise, for example, a deep learning neural network, or deepneural network (DNN), a generative adversarial network (GAN), a supportvector machine (SVM), e.g., a binary, non-binary, or multi-classclassifier, a linear or non-linear classifier, and so forth. In oneexample, the MLA may incorporate an exponential smoothing algorithm(such as double exponential smoothing, triple exponential smoothing,e.g., Holt-Winters smoothing, and so forth), reinforcement learning(e.g., using positive and negative examples after deployment as a MLM),and so forth. It should be noted that various other types of MLAs and/orMLMs may be implemented in examples of the present disclosure, such ask-means clustering and/or k-nearest neighbor (KNN) predictive models,support vector machine (SVM)-based classifiers, e.g., a binaryclassifier and/or a linear binary classifier, a multi-class classifier,a kernel-based SVM, etc., a distance-based classifier, e.g., a Euclideandistance-based classifier, or the like, and so on. In one example, atrained detection model may be configured to process those featureswhich are determined to be the most distinguishing features of theassociated item, e.g., those features which are quantitatively the mostdifferent from what is considered statistically normal or average fromother items that may be detected via a same system, e.g., the top 20features, the top 50 features, etc.

In one example, detection models (e.g., MLMs) may be trained and/ordeployed by AS 104 to process videos from cameras 132-134, cameras142-144, outward facing cameras of devices 131 and 141, or the like,and/or other input data to identify patterns in the features of thesensor data that match the detection model(s) for the respectiveitem(s). In one example, a match may be determined using any of thevisual features mentioned above, e.g., and further depending upon theweights, coefficients, etc. of the particular type of MLM. For instance,a match may be determined when there is a threshold measure ofsimilarity among the features of the video or other data streams(s) andan item/object signature. Similarly, in one example, AS 104 may apply anobject detection and/or edge detection algorithm to identify possibleunique items in video or other visual information (e.g., withoutparticular knowledge of the type of item; for instance, the object/edgedetection may identify an object in the shape of a tree in a videoframe, without understanding that the object/item is a tree). In thiscase, visual features may also include the object/item shape,dimensions, and so forth. In such an example, object recognition maythen proceed as described above (e.g., with respect to the “salient”portions of the image(s) and/or video(s)). In this regard, it should benoted that in one example, performance areas (such as fields of play inFIG. 1 , and in other examples, stages, pools, rinks, etc.) may havedesignated markings such that these locations are visually identifiableand may have an associated detection model that may detect suchlocations from images captured from various videos/camera feeds.

In one example, AS 104 may use such detection models to scan video feedsfrom cameras 142-144 of second physical environment 140 to determine abest matching video to a viewpoint of user 131 (e.g., as determined byapplying the same detection model(s) to identify features in a feed froman outward facing camera of device 131) and then selecting a camera witha video feed having the most matching features (e.g., in terms of theitems detected and the positions of such items). In one example, AS 104may also use such detection models to identify items or objects toextract, to include, to exclude, and so forth with regard to the visualcontent to be provided from camera 143 to device 131.

In one example, AS 104 may label visual markers within the visualcontent relating to a performance area, such as markers for detectedcorners of a field, markers for a detected center of a field, a markerfor a detected goal, goalpost, or the like, and so forth. For instance,these may comprise pixel coordinates of the field corners, and such.Thus, for example, device 131 may receive the visual content of camera143 from AS 104, may detect corresponding visual markers of theperformance area of first physical environment 130 within field of viewof user interface 135 of an outward facing camera of device 131, and maythen align the presentation of visual content from camera 143 (e.g., theAR content) using the received visual markers and the correspondingvisual markers detected in the first physical environment 130.

In one example, AS 104 may further train, possess, and/or applydetection models for detecting occurrences of particular occurrencetypes, such as a detection model for “excitement” in a stadium, adetection model for “excitement” of an announcer, “encroachment ofpersonal space”, etc. Such detection models may use visual features,including motion-based features in video, audio features, or both fordetecting various occurrences in accordance with the present disclosure.In addition, such detection models of the present disclosure may alsorelate to audio features including low-level audio features, including:spectral centroid, spectral roll-off, signal energy, mel-frequencycepstrum coefficients (MFCCs), linear predictor coefficients (LPC), linespectral frequency (LSF) coefficients, loudness coefficients, sharpnessof loudness coefficients, spread of loudness coefficients, octave bandsignal intensities, and so forth. Such audio features may also includehigh-level features, such as: words and phrases. In one example, one ormore detection models for one or more occurrence types may be deployedto device 131 for local use (e.g., to detect a situation where there isa danger to the user, or where an interruption in considered warrantedsuch that AR content should be made more transparent or should cease tobe presented).

The foregoing illustrates just one example of a system in which examplesof the present disclosure for obtaining and providing visual content ofa second venue to an augmented reality device of a user at a first venuewherein the visual content of the second venue is obtained from aviewpoint at the second venue that is matched to a user viewpoint of theuser at the first venue and/or for presenting visual content of a secondvenue as a visual overlay within a field of view of a user wherein thevisual content is from a viewpoint at the second venue that is matchedto a user viewpoint at a first venue may operate. In addition, althoughthe foregoing example(s) is/are described and illustrated in connectionwith a single viewer (user 191) at an event at first physicalenvironment 130 and with a single alternative venue (e.g., secondphysical environment 140), it should be noted that various otherscenarios may be supported in accordance with the present disclosure.For instance, user 191 may be provided with choices of multiplealternative venues from which AR content may be obtained and presented.In another example, user 191 may be present at the first physicalenvironment 130, but there may be no current event taking place at suchvenue, e.g., the home team is away playing a game at a different city.Nevertheless, user 191 (and other users) may experience one or moreother events at one or more other venues as AR content (e.g., othervenues of a same venue type), which may be live/current and/or which maybe historical, time-shifted events. In still another example, AS 104 mayestablish a voice or video call between users, such as user 191 and user192 via devices 131 and 141 to experience the same event at the sametime (e.g., the event at first physical environment 130, the event atsecond physical environment 140, or an event at a third venue whereneither user 191 nor user 192 is physically present). Although theviewing perspectives may be different, the occurrences within the eventmay be experienced at the same time and discussed contemporaneously.

It should also be noted that the system 100 has been simplified. Inother words, the system 100 may be implemented in a different form thanthat illustrated in FIG. 1 . For example, the system 100 may be expandedto include additional networks, and additional network elements (notshown) such as wireless transceivers and/or base stations, borderelements, routers, switches, policy servers, security devices, gateways,a network operations center (NOC), a content distribution network (CDN)and the like, without altering the scope of the present disclosure. Inaddition, system 100 may be altered to omit various elements, substituteelements for devices that perform the same or similar functions and/orcombine elements that are illustrated as separate devices.

As just one example, one or more operations described above with respectto server(s) 104 may alternatively or additionally be performed bydevice 131, and vice versa. In addition, although a single AS 104 isillustrated in the example of FIG. 1 , in other, further, and differentexamples, the same or similar functions may be distributed amongmultiple other devices and/or systems within the network 102, accessnetwork(s) 120 or 122, and/or the system 100 in general that maycollectively provide various services in connection with examples of thepresent disclosure for obtaining and providing visual content of asecond venue to an augmented reality device of a user at a first venuewherein the visual content of the second venue is obtained from aviewpoint at the second venue that is matched to a user viewpoint of theuser at the first venue and/or for presenting visual content of a secondvenue as a visual overlay within a field of view of a user wherein thevisual content is from a viewpoint at the second venue that is matchedto a user viewpoint at a first venue. Additionally, devices that areillustrated and/or described as using one form of communication (such asa cellular or non-cellular wireless communications, wiredcommunications, etc.) may alternatively or additionally utilize one ormore other forms of communication. Thus, these and other modificationsare all contemplated within the scope of the present disclosure.

To further aid in understanding the present disclosure, FIG. 2illustrates an example of detecting visual markers at a venue 200. Thevenue 200 may be the same or similar to the first physical environment130 of FIG. 1 (e.g., a football stadium). In accordance with the presentdisclosure, AR devices, such as devices 131 and 141 of FIG. 1 , may beconfigured to detect particular visual markers associated withparticular venue types. It should be noted that an AR device may alsocomprise a mobile endpoint device, such as a smartphone, performing thesame operations, wherein the presentation to the user is accomplishedvia a coupled AR headset (e.g., where the headset performs more limitedfunctions of displaying visual AR content (and in some cases audiocontent), reporting orientation information, providing captured videoand/or audio, etc.). To illustrate, football fields are generallystandardized and include the same basic features of 100 yards lengthplus endzones, goalposts, etc. Similarly, baseball fields typicallyinclude three bases, a home plate, a pitcher's mound, base lines/foulball lines, and in some cases, foul poles, outfield walls/fences,infield dirt, etc. Likewise, theaters may generally include a stage andcurtains framing the stage, and so on for various other types of venues.Thus, for example, a user may provide an input indicating the type ofvenue, or an AR device may determine the type of venue viadetection/recognition models for detecting different types of eventvenues, via electronic calendar or other information accessible to anapp running on a user's AR device, and so on. In one example, based onthe venue type of a venue at which the AR device is physically present,an AR device may then determine particular visual markers that may beused to align AR content from another venue. In the example of FIG. 2 ,the venue 200 is a football stadium. Accordingly, in an illustrativeexample, visual markers 210 that may be used to align AR content maycomprise field borders, endzone lines, center line, center circle,and/or goalposts. In one example, the AR device may provide these visualmarkers to an AR content server (such as 104 of FIG. 1 ) for use inmatching a user viewpoint at venue 200 to a particular available cameraviewpoint at another venue. Alternatively, or in addition, the AR devicemay use the visual markers to align AR content (e.g., visual content)from another venue as overlay content within a field of view of the ARdevice.

FIG. 3 illustrates an example view 300, which may be a non-AR view ofvenue 200 of FIG. 2 during an event, e.g., a football game between anAtlanta team and a Los Angeles team. FIG. 4 illustrates an example view400, which may represent the visual experience of a user at the venue200 who is physically present during the event illustrated in FIG. 3 ,but who is currently experiencing AR content from another venue via anAR device of the user. For instance, as illustrated in FIG. 4 , the ARdevice may present, as an overlay 410, visual content from anotherfootball game between a New York team and a Philadelphia team takingplace at another venue. As can be seen in FIG. 4 , the overlay 410 maycomprise a dominant overlay (e.g., a visually dominant overlay), suchthat the AR content may be primarily visible, while the real-worldimagery of the venue 200 remains visible with the overlay 410 havingsome level of transparency.

It should be noted that in one example, the AR device of the user mayreceive the AR content from the other venues, and may use the visualmarkers previously identified to align the AR content with the currentview of the user based upon orientation of the AR device. For instance,the AR device may identify visual markers in the AR content (e.g., thesame or similar visual markers 210 identified in the real-world view ofvenue 200 illustrated in FIG. 2 ; in this case, end lines, goal posts,etc.). Alternatively, or in addition, the visual markers in the ARcontent may be included as additional information provided to the ARdevice along with the AR content (e.g., from an AR content server).Thus, the AR device may determine the correct position and/ororientation in which to project the AR content as overlay 410 and/orwithin overlay 410 such that the visual markers of the AR content are inthe same positions as the corresponding visual markers in the real-worldimagery of venue 200.

The example of FIG. 4 further illustrates that in one example, the ARdevice and/or AR content server may also identify additional visibleregions that may be used to project event information. For example, theAR device and/or AR content server may detect a scoreboard 420, e.g.,using an object detection/recognition model for “scoreboard,” and mayuse this region to project a virtual scoreboard 425 for informationrelating to the event at the other venue (e.g., the score from the NewYork-Philadelphia game, the time remaining in the game, etc.). Theprojection of virtual scoreboard 425 may involve aligning in the same orsimilar manner as the AR content presented in overlay 410.

FIG. 5 illustrates an alternative example view 500, which may be similarto the view 400, but in which a virtual scoreboard 525 is presented inaddition to AR content in an overlay 510 (which may be the same orsimilar to overlay 410 of FIG. 4 ). In this case, a region within afield of view of the user may be selected for presentation of thevirtual scoreboard 525. For instance, the AR device may select a regionthat is determined to be “unimportant” or “available for projection.”For example, other regions in the field of view may be determined to be“important” according to defined rules, e.g., detected area ofperformance, scoreboard(s), etc. may be designated as “important” and/or“not available for virtual scoreboard projection, etc. Thus, forinstance, the user may perceive the scoreboard 520 with a view that isunobstructed, may see that play has resumed at the live/in-person eventat venue 200, and may choose to switch back to watching thelive/in-person event.

FIG. 6 illustrates a flowchart of an example method 600 for obtainingand providing visual content of a second venue to an augmented realitydevice of a user at a first venue wherein the visual content of thesecond venue is obtained from a viewpoint at the second venue that ismatched to a user viewpoint of the user at the first venue. In oneexample, steps, functions and/or operations of the method 600 may beperformed by a device or apparatus as illustrated in FIG. 1 , e.g., byAS 104, or any one or more components thereof, or by AS 104, and/or anyone or more components thereof in conjunction with one or more othercomponents of the system 100, such as device 131, camera(s) 142-144, andso forth. In one example, the steps, functions, or operations of method600 may be performed by a computing device or processing system, such ascomputing system 800 and/or hardware processor element 802 as describedin connection with FIG. 8 below. For instance, the computing system 800may represent any one or more components of the system 100 that is/areconfigured to perform the steps, functions and/or operations of themethod 600. Similarly, in one example, the steps, functions, oroperations of the method 600 may be performed by a processing systemcomprising one or more computing devices collectively configured toperform various steps, functions, and/or operations of the method 600.For instance, multiple instances of the computing system 800 maycollectively function as a processing system. For illustrative purposes,the method 600 is described in greater detail below in connection withan example performed by a processing system. The method 600 begins instep 605 and proceeds to step 610.

At step 610, the processing system identifies a user viewpoint of a userat a first venue, e.g., via an augmented reality (AR) device, or ARendpoint device. For instance, the user viewpoint may comprise aposition of the AR device (e.g., also comprising the position of theuser) relative to a performance area of the venue (e.g., in relation toone or more locational/positional markers of venue). In one example, thelocational/positional markers may be identified by seat number, row,section, etc., may be identified by geographic coordinates and/orcoordinates with respect to a reference coordinate system of the venue,and so on. For instance, a distance and bearing of the location of theuser to a particular locational marker of the first venue may be knownor determined (e.g., a distance and bearing to a center of a field ofplay, a distance and bearing to a closest goal post, etc.), or similarlyx,y vector component distances (or x,y,z where a height of viewingand/or seating position is relevant). In another example, the userviewpoint may be represented as locations of visual markers of aperformance area of the venue within one or more images of the venuefrom a field of view of the AR device. For example, the visual markersmay be such as illustrated in FIG. 2 and discussed above, or the like.In one example, the visual markers may be provided by the AR device, orthe processing system may obtain visual data from the AR device (e.g.,image(s) and/or video from an outward facing camera) from which theprocessing system may detect visual markers via one or more detectionmodels such as discussed above.

At step 620, the processing system matches a viewpoint at a second venueto the user viewpoint of the user at the first venue, e.g., where anevent is occurring at the second venue to be viewed by the user asvisual content, or AR content, via an AR device of the user at the firstvenue. In one example, step 620 may comprise matching a first positionof the user viewpoint of the user at the first venue to a secondposition of the viewpoint of the second venue. In accordance with thepresent disclosure, the first venue and the second venue may be of asame venue type. Thus, in one example, the first venue and the secondvenue may have corresponding areas of performance. In one example, adistance and bearing, or vector component distances of a camera withrespect to a same locational/positional marker at a second venue may beknown and recorded in a venue map of the second venue. Thus, the closestmatching viewpoint may be a viewpoint of a camera that is the closestrelative distance (e.g., closest vector distance of the respectivecamera and user viewpoint/location). For instance, the user viewpointmay comprise a first position at the first venue and the viewpoint ofthe second venue may comprise a second position at the second venue. Inone example, the first position may comprise a first seat at the firstvenue and the second position may comprise a second seat at the secondvenue. In one example, step 620 may include determining that the firstseat position and the second seat position have a same orientation anddistance to corresponding performance areas of the first venue and thesecond venue (e.g., the first seat has a first orientation and a firstdistance to a first performance area of the first venue, the second seathas a second orientation and a second distance to the second performancearea of the second venue, the first orientation and the secondorientation are the same, the first distance and the second distance arethe same, and the first performance area and the second performance areaare of a same type and have same dimensions, or as close thereto asavailable). In one example, the matching of step 620 may be viarespective venue maps of the first venue and the second venue.

In another example, step 620 may alternatively or additionally comprisevisual analysis of video from cameras at the second venue to determine abest matching viewpoint (e.g., of one of the available cameras). Forinstance, step 620 may comprise aligning at least one visual marker ofthe areas of performance (e.g., aligning at least one visual marker ofthe first venue and at least one corresponding visual marker of thesecond venue). Thus, for example, the at least one visual marker of thefirst venue may be obtained from a visual feed video from the augmentedreality device of the user, and the at least one corresponding visualmarker of the second venue may be obtained from at least one device atthe second venue that provides the visual content of the second venue(e.g., at least one camera, a server aggregating video feeds frommultiple cameras at the second venue, etc.). To illustrate, theprocessing system may scan video feeds from cameras of the second venueto determine a best matching video to the user viewpoint (e.g., asdetermined by applying the same detection model(s) to identify featuresin a feed from an outward facing camera of the AR device and selecting acamera with a video feed having the most matching features (e.g., interms of the items detected and the positions of such items)).

In one example, the at least one visual marker may comprise at least oneof: a field line, at least a portion of a goal (such as a goal post,crossbar, etc.), a wall, a portion of an infield, a foul post, a rinkboard, a lane line (e.g., for a track, swimming, or rowing venue, forexample), a backboard and/or a hoop (e.g., for a basketball venue), atleast one net (e.g., a tennis net, a volleyball net, etc.), at least onetrack feature (such as curves and straightaways, starting line, etc. foreither track and field, car racing, etc.), and so on. For instance, thefirst and second venues may be of a sporting venue type (e.g., abaseball field and/or stadium, a football field and/or stadium, a hockeyrink and/or arena, a pool, etc.). In another example, the first venueand the second venue have corresponding areas of performance comprisingstages (e.g., of a theater or the like). In such an example, the atleast one visual marker may comprise at least one feature shared by thecorresponding stages (such as a front edge, the stage ends, which may beidentified by where the stage edge meets curtains, etc.). It shouldagain be noted that the event at the second venue may be contemporaneous(live), or may be recorded and made available to the user in atime-shifted manner. For example, the first venue may comprise a localsports field where the user may watch a historical professional sportingevent.

At step 630, the processing system detects a trigger condition toprovide visual content of the second venue to the user at the firstvenue. In one example, the trigger condition may comprise a user inputvia the AR device. In one example, the user input may also include adefined duration of the switch to viewing an event at the second venueand/or a trigger condition for switching back to a live/in-person viewof the first venue, e.g., an exciting play at the first venue, an end ofa time out, etc. In one example, the trigger condition may comprise anoccurrence of a defined occurrence type at one of the first venue or thesecond venue (e.g., a beginning or end of a time out, a start or end ofa period, half, or quarter, an exciting play, etc.). Thus, in variousexamples the trigger condition may be detected from: at least one ofaudio data of the first venue or video data of the first venue, at leastone of audio data of the second venue or video data of the second venue,an announcement at the first venue or the second venue, a voice patternof an announcer at the first venue or the second venue, and so on. Itshould be noted that audio or visual data of first venue and/or secondvenue for detecting the occurrence do not necessarily need to come fromfirst viewpoint and second viewpoint, but may use any general camera ormicrophone for the entire venue, or a handful of cameras and/ormicrophones.

Examples of trigger conditions may include, a stop in play, a start inplay, a volume exceeding a first threshold, a volume below a secondthreshold, a sound pattern of the occurrence type, a voice pattern of anannouncer indicative of a current importance of the first event or thesecond event, an announcement indicative of a break in the first eventor the second event, an announcement indicative of a start of the firstevent or the second event, an announcement indicative of a resumption ofthe first event or the second event after a break, visual dataindicative of a break, a start, or a resumption of the first event orthe second event, visual data indicative of a current importance of thefirst event or the second event, and so forth. It should be noted thatthe present method 600 does not require that there be an event (such asa game, concert, performance, etc.) at the first venue. However, innumerous examples there may be an event occurring at the first venuethat may affect whether and when the AR device may be switched between aview of AR content of the second venue and a live/in-person view of thefirst venue.

At step 640, the processing system obtains the visual content of thesecond venue, where the visual content of the second venue is obtainedfrom the viewpoint at the second venue (that is matched to the userviewpoint of the user at the first venue). For instance, step 640 mayinclude obtaining video from a camera of the second venue having aclosest corresponding viewpoint the user viewpoint (or visual contentderived from such video).

At optional step 650, the processing system may detect at least onecorresponding visual marker of the second venue in the visual data ofthe second venue. For instance, optional step 650 may include detectingpositions within the visual data of field lines, goal posts, stageedge(s), etc., in the same or similar manner as discussed above.

At step 660, the processing system provides the visual content of thesecond venue to an AR device of the user at the first venue, where theAR device presents the visual content of the second venue as a visualoverlay within a field of view of the user. For instance, the AR devicemay present the visual content as a visual overlay when the user looksin the direction of a play/performance area of the first venue having acorresponding play/performance area of the second from which the visualcontent is captured from the viewpoint at the second venue. In oneexample, the AR device may align at least one visual marker of the firstvenue and at least one corresponding visual marker of the second venuefor the presenting. In one example, step 660 may include providing theat least one corresponding visual marker that may be determined atoptional step 650 to the AR device.

Following step 660, the method 600 proceeds to step 695. At step 695,the method 600 ends.

It should be noted that the method 600 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example, the processing systemmay repeat one or more steps of the method 600, such as step 610-660 fordifferent viewing locations, different users, etc., steps 620-660 forthe same user at a different time during a same visit to the firstvenue, and so on. In one example, the method 600 may comprise performingstep 630 prior to step 610, e.g., wherein step 610 and subsequent stepsmay be performed in response to the detecting of the trigger condition.In one example, the method 600 may include obtaining authorization togather video of the first venue from the AR device of the user, whichcan be provided to others seeking to view AR content from the firstvenue while at the second venue (or another venue). In this regard, itshould be noted that one or more cameras of the second venue discussedabove may also comprise user devices that may be opted-in to visualcontent sharing (and in one example, accompanying audio data sharing).In one example, the method 600 may include providing a choice ofavailable venues from which to obtain AR content and obtaining a userselection of the second venue.

It should be noted that “visual content” of the second venue may bevideo from a camera having a closest corresponding viewpoint to the userviewpoint, but may also comprise extracted portions of the video. Thus,“video data” of the second venue that may be used at step 630 could bethe same visual data that is provided to the user at the first venue atstep 660, the same video from which visual data comprising less than allof the video is extracted and provided to the user, or could be adifferent video (e.g., from a different camera). In various otherexamples, the method 600 may further include or may be modified tocomprise aspects of any of the above-described examples in connectionwith FIGS. 1-5 , or as otherwise described in the present disclosure.Thus, these and other modifications are all contemplated within thescope of the present disclosure.

FIG. 7 illustrates a flowchart of an example method 700 for presentingvisual content of a second venue as a visual overlay within a field ofview of a user wherein the visual content is from a viewpoint at thesecond venue that is matched to a user viewpoint at a first venue. Inone example, steps, functions and/or operations of the method 700 may beperformed by a device or apparatus as illustrated in FIG. 1 , e.g., bydevice 131, or any one or more components thereof, or by device 131,and/or any one or more components thereof in conjunction with one ormore other components of the system 100, such as 104, camera(s) 142-144,and so forth. In one example, the steps, functions, or operations ofmethod 700 may be performed by a computing device or processing system,such as computing system 800 and/or hardware processor element 802 asdescribed in connection with FIG. 8 below. For instance, the computingsystem 800 may represent any one or more components of the system 100that is/are configured to perform the steps, functions and/or operationsof the method 700. Similarly, in one example, the steps, functions, oroperations of the method 700 may be performed by a processing systemcomprising one or more computing devices collectively configured toperform various steps, functions, and/or operations of the method 700.For instance, multiple instances of the computing system 800 maycollectively function as a processing system. For illustrative purposes,the method 700 is described in greater detail below in connection withan example performed by a processing system. The method 700 begins instep 705 and proceeds to step 710.

At step 710, the processing system (e.g., of a user endpointdevice/augmented reality device) detects a trigger condition to providea visual content of a second venue to a user at a first venue. As notedabove, the first venue and the second venue may be of a same venue type.Accordingly, the first venue and the second venue may have correspondingareas of performance (e.g., stages for performance, fields, rinks,pools, etc.). The trigger condition may comprise a user input (via theaugmented reality device), or an occurrence of a defined occurrence typeat one of the first venue or the second venue. For instance, theoccurrence may be detected via one or more detection models in the sameor similar manner as discussed above. In one example, step 710 maycomprise the same or similar operations as step 630 of the examplemethod 600 of FIG. 6 discussed above (e.g., however performed by theprocessing system of the augmented reality device in connection with themethod 700).

At step 720, the processing system identifies a user viewpoint of theuser at the first venue. In one example, step 720 may comprise the sameor similar operations as step 610 of the example method 600 of FIG. 6discussed above (e.g., however performed by the processing system of theaugmented reality device in connection with the method 700).

At optional step 730, the processing system may match a viewpoint at thesecond venue (e.g., a camera location/position and/or a view from acamera of a plurality of cameras at the second venue) to the userviewpoint of the user at the first venue. It should be noted that step730 may be optional insofar as, in one example, the matching may beperformed by a network-based processing system or server, e.g., an ARcontent server. In one example, optional step 730 may comprise the sameor similar operations as step 620 of the example method 600 of FIG. 6discussed above (e.g., however performed by the processing system of theaugmented reality device in connection with the method 700). Forinstance, in one example, the matching may comprise aligning at leastone visual marker of the areas of performance.

At step 740, the processing system obtains the visual content of thesecond venue, where the visual content of the second venue is obtainedfrom a viewpoint at the second venue that is matched to the userviewpoint of the user at the first venue. In one example, the visualcontent may be obtained from a camera of the second venue (e.g., havingthe matched viewpoint/location) via a network-based processing system orserver, e.g., an AR content server.

At optional step 750, the processing system may detect at least onevisual marker of the first venue. For instance, the at least one visualmarker may be detected in images/video from a camera of the processingsystem (e.g., an outward facing camera of the AR device of theprocessing system). In one example, optional step 750 may comprise theperformance of operations similar to step 620 of the example method 600of FIG. 6 (e.g., however performed by the processing system of theaugmented reality device in connection with the method 700)

At optional step 760, the processing system may detect at least onecorresponding visual marker of the second venue in the visual content ofthe second venue. For instance, the at least one corresponding visualmarker may be detected in the same or similar manner as the at least onevisual marker is detected at optional step 750. In one example, optionalstep 760 may comprise the performance of operations similar to step 650of the example method 600 of FIG. 6 (e.g., however performed by theprocessing system of the augmented reality device in connection with themethod 700).

At step 770, the processing system presents the visual content of thesecond venue as a visual overlay within a field of view of the user. Inone example, step 770 may include aligning at least one visual marker ofthe first venue and at least one corresponding visual marker of thesecond venue. For instance, the visual content may be projected astransparent (but dominant) imagery via angled projection on a glass orsimilar screen within the user's field of view. In one example, thetransparency may be selected such that the user will primarily perceivethe AR content (visual content from the second venue), while thereal-world aspects of the first venue may remain slightly visible in thecoincident portions of the overlay.

Following step 770, the method 700 proceeds to step 795. At step 795,the method 700 ends.

It should be noted that the method 700 may be expanded to includeadditional steps, or may be modified to replace steps with differentsteps, to combine steps, to omit steps, to perform steps in a differentorder, and so forth. For instance, in one example, the processing systemmay repeat one or more steps of the method 700, such as step 710-760 fordifferent viewing locations, for a different time during a same visit tothe first venue, for the same user at a different venue, and so on. Inone example, the method 700 may be modified to perform step 720 prior tostep 710. In one example, the method 700 may include detecting a triggercondition for switching back to viewing an in-person event at the firstvenue (or simply viewing in-person real-world imagery of the firstvenue, if there is no current event taking place at the first venue),and ceasing the presentation of the visual content from the second venuein response thereto. In one example, the method 700 may includeselecting and or adjusting a transparency of the overlay, e.g., inresponse to changing light levels or other conditions. In one example,the method 700 may include obtaining authorization to gather and sharevideo of the first venue (e.g., as AR content for others seeking to viewan event at the first venue while physically present at the second venue(or another venue)), and providing video or visual content derivedtherefrom, in response to the authorization. In one example, the method700 may include providing a choice of available venues from which toobtain AR content and obtaining a user selection of the second venue. Inanother example, the method 700 may include identifying usable areas forprojection, e.g., a field of play and any scoreboard(s) of the firstvenue that may be used for projecting video from a field of play for theevent at the second venue and additional information, such as scores,time remaining, etc., for the event at the second venue as a virtualscoreboard, for instance. In one example, the method 700 may includepresenting additional AR content comprising an additional overlay withinformation from one or more other venues (e.g., a third venue, etc.).In various other examples, the method 700 may further include or may bemodified to comprise aspects of any of the above-described examples inconnection with FIGS. 1-6 , or as otherwise described in the presentdisclosure. Thus, these and other modifications are all contemplatedwithin the scope of the present disclosure.

In addition, although not expressly specified above, one or more stepsof the method 600 or the method 700 may include a storing, displayingand/or outputting step as required for a particular application. Inother words, any data, records, fields, and/or intermediate resultsdiscussed in the method(s) can be stored, displayed and/or outputted toanother device as required for a particular application. Furthermore,operations, steps, or blocks in FIGS. 6 and 7 that recite a determiningoperation or involve a decision do not necessarily require that bothbranches of the determining operation be practiced. In other words, oneof the branches of the determining operation can be deemed as anoptional step. However, the use of the term “optional step” is intendedto only reflect different variations of a particular illustrativeembodiment and is not intended to indicate that steps not labelled asoptional steps to be deemed to be essential steps. Furthermore,operations, steps or blocks of the above described method(s) can becombined, separated, and/or performed in a different order from thatdescribed above, without departing from the example embodiments of thepresent disclosure.

FIG. 8 depicts a high-level block diagram of a computing system 800(e.g., a computing device or processing system) specifically programmedto perform the functions described herein. For example, any one or morecomponents, devices, and/or systems illustrated in FIG. 1 or describedin connection with FIGS. 2-7 , may be implemented as the computingsystem 800. As depicted in FIG. 8 , the computing system 800 comprises ahardware processor element 802 (e.g., comprising one or more hardwareprocessors, which may include one or more microprocessor(s), one or morecentral processing units (CPUs), and/or the like, where the hardwareprocessor element 802 may also represent one example of a “processingsystem” as referred to herein), a memory 804, (e.g., random accessmemory (RAM), read only memory (ROM), a disk drive, an optical drive, amagnetic drive, and/or a Universal Serial Bus (USB) drive), a module 805for obtaining and providing visual content of a second venue to anaugmented reality device of a user at a first venue wherein the visualcontent of the second venue is obtained from a viewpoint at the secondvenue that is matched to a user viewpoint of the user at the first venueand/or for presenting visual content of a second venue as a visualoverlay within a field of view of a user wherein the visual content isfrom a viewpoint at the second venue that is matched to a user viewpointat a first venue, and various input/output devices 806, e.g., a camera,a video camera, storage devices, including but not limited to, a tapedrive, a floppy drive, a hard disk drive or a compact disk drive, areceiver, a transmitter, a speaker, a display, a speech synthesizer, anoutput port, and a user input device (such as a keyboard, a keypad, amouse, and the like).

Although only one hardware processor element 802 is shown, the computingsystem 800 may employ a plurality of hardware processor elements.Furthermore, although only one computing device is shown in FIG. 8 , ifthe method(s) as discussed above is implemented in a distributed orparallel manner for a particular illustrative example, e.g., the stepsof the above method(s) or the entire method(s) are implemented acrossmultiple or parallel computing devices, then the computing system 800 ofFIG. 8 may represent each of those multiple or parallel computingdevices. Furthermore, one or more hardware processor elements (e.g.,hardware processor element 802) can be utilized in supporting avirtualized or shared computing environment. The virtualized computingenvironment may support one or more virtual machines which may beconfigured to operate as computers, servers, or other computing devices.In such virtualized virtual machines, hardware components such ashardware processors and computer-readable storage devices may bevirtualized or logically represented. The hardware processor element 802can also be configured or programmed to cause other devices to performone or more operations as discussed above. In other words, the hardwareprocessor element 802 may serve the function of a central controllerdirecting other devices to perform the one or more operations asdiscussed above.

It should be noted that the present disclosure can be implemented insoftware and/or in a combination of software and hardware, e.g., usingapplication specific integrated circuits (ASIC), a programmable logicarray (PLA), including a field-programmable gate array (FPGA), or astate machine deployed on a hardware device, a computing device, or anyother hardware equivalents, e.g., computer-readable instructionspertaining to the method(s) discussed above can be used to configure oneor more hardware processor elements to perform the steps, functionsand/or operations of the above disclosed method(s). In one example,instructions and data for the present module 805 for obtaining andproviding visual content of a second venue to an augmented realitydevice of a user at a first venue wherein the visual content of thesecond venue is obtained from a viewpoint at the second venue that ismatched to a user viewpoint of the user at the first venue and/or forpresenting visual content of a second venue as a visual overlay within afield of view of a user wherein the visual content is from a viewpointat the second venue that is matched to a user viewpoint at a first venue(e.g., a software program comprising computer-executable instructions)can be loaded into memory 804 and executed by hardware processor element802 to implement the steps, functions or operations as discussed abovein connection with the example method(s). Furthermore, when a hardwareprocessor element executes instructions to perform operations, thiscould include the hardware processor element performing the operationsdirectly and/or facilitating, directing, or cooperating with one or moreadditional hardware devices or components (e.g., a co-processor and thelike) to perform the operations.

The processor (e.g., hardware processor element 802) executing thecomputer-readable instructions relating to the above described method(s)can be perceived as a programmed processor or a specialized processor.As such, the present module 805 for obtaining and providing visualcontent of a second venue to an augmented reality device of a user at afirst venue wherein the visual content of the second venue is obtainedfrom a viewpoint at the second venue that is matched to a user viewpointof the user at the first venue and/or for presenting visual content of asecond venue as a visual overlay within a field of view of a userwherein the visual content is from a viewpoint at the second venue thatis matched to a user viewpoint at a first venue (including associateddata structures) of the present disclosure can be stored on a tangibleor physical (broadly non-transitory) computer-readable storage device ormedium, e.g., volatile memory, non-volatile memory, ROM memory, RAMmemory, magnetic or optical drive, device or diskette and the like.Furthermore, a “tangible” computer-readable storage device or medium maycomprise a physical device, a hardware device, or a device that isdiscernible by the touch. More specifically, the computer-readablestorage device or medium may comprise any physical devices that providethe ability to store information such as instructions and/or data to beaccessed by a processor or a computing device such as a computer or anapplication server.

While various examples have been described above, it should beunderstood that they have been presented by way of example only, and notlimitation. Thus, the breadth and scope of a preferred example shouldnot be limited by any of the above-described examples, but should bedefined only in accordance with the following claims and theirequivalents.

What is claimed is:
 1. A method comprising: detecting, by a processingsystem comprising at least one processor, a trigger condition to providea visual content of a second venue to a user at a first venue, whereinthe apparatus comprises an augmented reality device of the user;identifying, by the processing system, a user viewpoint of the user atthe first venue; obtaining, by the processing system, the visual contentof the second venue, wherein the visual content of the second venue isobtained from a viewpoint at the second venue that is matched to theuser viewpoint of the user at the first venue; and presenting, by theprocessing system, the visual content of the second venue as a visualoverlay within a field of view of the user.
 2. The method of claim 1,wherein the first venue and the second venue are of a same venue type.3. The method of claim 2, wherein the first venue and the second venuehave corresponding areas of performance.
 4. The method of claim 1,further comprising: matching, by the processing system, the viewpoint atthe second venue to the user viewpoint of the user at the first venue.5. The method of claim 1, wherein the presenting comprises: aligning atleast one visual marker of the first venue and at least onecorresponding visual marker of the second venue.
 6. The method of claim5, wherein the at least one visual marker comprises at least one of: atleast one field line; at least a portion of a goal; at least one wall;at least a portion of an infield; at least one foul ball post; at leastone rink board; at least one lane line; at least one net; at least onebackboard; at least one hoop; or at least one track feature.
 7. Themethod of claim 1, wherein the trigger condition comprises an occurrenceof a defined occurrence type at one of: the first venue or the secondvenue.
 8. The method of claim 7, wherein the trigger condition isdetected from: at least one of: audio data of the first venue or videodata of the first venue; at least one of: audio data of the second venueor video data of the second venue; an announcement at the first venue orthe second venue; or a voice pattern of an announcer at the first venueor the second venue.
 9. The method of claim 1, wherein the userviewpoint of the user at the first venue is forwarded to a networkserver over a communications network.
 10. The method of claim 9, whereinthe visual content of the second venue is obtained from the networkserver over the communications network based on the user viewpoint. 11.The method of claim 5, wherein the first venue and the second venue havecorresponding stages for performance, wherein the at least one visualmarker comprises at least one feature shared by the correspondingstages.
 12. The method of claim 5, wherein the at least one visualmarker of the first venue is obtained from a video from the augmentedreality device of the user, wherein at least one corresponding visualmarker of the second venue is obtained from at least one device locatedat the second venue that provided the visual content of the secondvenue.
 13. The method of claim 1, wherein the trigger conditioncomprises a user input via the augmented reality device.
 14. The methodof claim 1, wherein the viewpoint at the second venue that is matchedcomprises: matching a first position of the user viewpoint of the userat the first venue to a second position of the viewpoint of the secondvenue.
 15. The method of claim 14, wherein the first position comprisesa first seat at the first venue, and wherein the second positioncomprises a second seat at the second venue.
 16. The method of claim 14,wherein first position and the second position have a same orientationand distance to corresponding performance areas of the first venue andthe second venue.
 17. A non-transitory computer-readable medium storinginstructions which, when executed by a processing system including atleast one processor, cause the processing system to perform operations,the operations comprising: detecting, a trigger condition to provide avisual content of a second venue to a user at a first venue, wherein theapparatus comprises an augmented reality device of the user; identifyinga user viewpoint of the user at the first venue; obtaining the visualcontent of the second venue, wherein the visual content of the secondvenue is obtained from a viewpoint at the second venue that is matchedto the user viewpoint of the user at the first venue; and presenting thevisual content of the second venue as a visual overlay within a field ofview of the user.
 18. An apparatus comprising: a processing systemincluding at least one processor; and a computer-readable medium storinginstructions which, when executed by the processing system, cause theprocessing system to perform operations, the operations comprising:detecting, a trigger condition to provide a visual content of a secondvenue to a user at a first venue, wherein the apparatus comprises anaugmented reality device of the user; identifying a user viewpoint ofthe user at the first venue; obtaining the visual content of the secondvenue, wherein the visual content of the second venue is obtained from aviewpoint at the second venue that is matched to the user viewpoint ofthe user at the first venue; and presenting the visual content of thesecond venue as a visual overlay within a field of view of the user. 19.The apparatus of claim 18, wherein the first venue and the second venueare of a same venue type.
 20. The apparatus of claim 18, the operationsfurther comprising: matching the viewpoint at the second venue to theuser viewpoint of the user at the first venue.